PRODUCT BLOG

a deeper look at conversion #HowToManageAProduct

Let me tell you the story of my friend Peter*.

Peter is working at a tech company, called findmyho.me as Head of Product. Their product is a real estate search app. The users can search for their new homes to buy. When they find something that they like, they can contact the listing agent using the product. The revenue is coming from the agents. The agent pays them if a potential customer contacts them. And also if the real estate is eventually sold then they get a flat rate of the commission. The product is available both on the web and you can download the app both for iOS and Android.

 
 

 

The main KPI of the company is the number of users who contact the agent out of the people who start using the product. 

Let's say that 1000 people started using the product on a certain day. Out of these people 150 people contacted the agent on the same day. So the conversion rate is 15%.

 
 

This conversion varies across platforms. On the web it's 19% and in the app is 11%. Based on this it was clear that people are more likely to contact the agent using the web than the app. It's a clear decision to invest more into the web if the business wants to make more money.

For Peter this was counterintuitive. Most of the time he talked with the users he heard that they were using the app and not the web to contact the agent. So how is it possible that they converted less?

He looked into the analytics deeper and found the answer on this chart:

This chart tells us that the app converts worse in the first 38 days, but after that it massively beats the web. In other words: people who start using the product on the web are more likely to convert in the first month. On a longer term though people who started using the app are way more likely to convert.

This was the point when Peter realized that there is a serious issue in the company. An issue that can be deadly to the whole business and stems from one small thing that was overlooked in the beginning.

One Small Thing

Usually when we look at the main funnels of our product we tend to overlook one thing. This tiny thing is one of the most important settings in product analytics.

On top of it, this small thing is the connection between conversion and retention. Understanding this is crucial in the field of product management.

This small thing is the Funnel Window Size.

In this post I'm going to write about the Funnel Window Size and a related topic: conversion characteristics. It's a deeper look at conversion.

Funnel Windows

Let's look at the basic funnel that describes findmyho.me app:

In this funnel we give 1 day to the users to convert (to contact the agent). We measure the conversion rate based on this. Approx. 11% in the app. But what if we would give more time to the users to convert. 7 days for example:

Or maybe 30 days:

The conversion rate is increasing if we increase the funnel window size. Well, you could say, this wasn't that surprising. It's a monotonically increasing number, since users cannot "unconvert" as time goes by.

So then why is this so important? 

Let's think about it. What do we expect from people who are planning to buy a new home? To find a home in 1 day or to spend some time with research and discuss this with their family. Clearly the second one is more realistic based on my observations and experience. It takes time to make a decision to spend serious amount of money on a new home. Definitely not a one day "project". 

This means that a funnel with a one day funnel window is not representing the people's behavior.

We looked at the conversion rate for 7 and 30 days windows above. Let's look at it for a lot more different days. Let's look at how the conversion changes over time.

Conversion Characteristics

The following curve shows the conversion by different funnel window sizes for the same cohort of users in the app:

This is the kind of curve that Peter looked at in the beginning.

Since we can segment the users many ways, conversion curves can be analysed by various segments.

For example besides comparing platform performances we could also segment by user acqusition sources:

We can clearly see the surprising fact. Display Advertising seems to be the worst in the beginning but eventually it beats both SEM and Social. There is also an interesting learning that people who are coming from the partners of findmyho.me seem to convert worse than the people who are coming from Google and other search engines. But on the long term people coming from partners convert better.

We can also compare the various markets

We can find important learnings here too. These can be attributed either to cultural behavioural differences or market strategy differences. In both cases it gives us more insight about where to change things. For example in South America in the beginning the conversion is very low, but then it increases rapidly. If the market strategy of findmyho.me is the same across all markets then it could mean that people in South America take more time to research and discuss the actual purchasing decisions with their families compared to Europe.

As you can see we can extract tons more learnings from the conversion curves compared to a single number. These curves describe the behaviour of our users and the performance of our business too.

This was the thing that Peter realised. Picking a wrong a funnel window can be deadly to the business. Why?

Wrong KPI

Because it's easy to say that the most important KPI is the conversion. And it will stick. Everyone will blindly talk about how to increase the conversion rate from 15% to 16%, or with X%, etc. But if the conversion funnel has an inappropriate window size then it's not representing the people whom we are building the product.

 
 

This means that we picked the wrong number. And it has a terrible amplifying effect. People start optimising for this KPI. For example in case of findmyho.me they want more people to convert in one day. This results in meaningless product changes aiming for only increasing the conversion rate but not creating real value for people. And if we don't create value for people then they will leave us.

Let's illustrate it with a simplified example. To increase the 1 day conversion rate, you could introduce a feature that notifies the agent automatically to call any new user if they looked at a listed house in the product. This would result in almost a 100% one day conversion rate. On the other hand people would get so annoyed that they would immediately leave the product. Eventually the number of users would decrease (conversion would still stay at a 100% level), the product would die and so the business.

To put it bluntly findmyho.me employees would confuse the KPI with the Goal. I have seen this many times.

The goal in case of findmyho.me is to help people find their new homes they are looking for. We know from behavioural studies that the more expensive a thing is the more time it will take to make a purchasing decision. A new home falls under this category.

Don't get me wrong. It's important to improve the conversion rate. But it really matters what conversion rate exactly. In case of findmyho.me the right goal could be that every single person who started looking for a home contacted an agent in 30 days. This means a 100% conversion rate with a 30 days funnel window size. This is amazingly ambitious I know. To solve this we need to think about the whole user journey and not just optimise for a short term conversion rate. In this case we need to think about cross-platform usage and not just about one platform. If we think like this we will truly create a valuable product for the people who are searching for a new home. If we don't do this we are very likely to transform the product into a faceless transactional service.

 
 

So what to do?

  • Define your goal first
  • Revisit your funnels and check if the window sizes are reflecting the goal. If not change them.
  • Look at the conversion curves for the funnels
  • Build a strategy how to change these curves exactly (where to lift them)
  • Whenever you design and develop a feature consider all the above when picking the success metrics

Talking about the user journey and value...

It may ring the bell for growth hackers too. One can use retention to describe these. In my next post I will show how conversion actually equals retention via the conversion characteristics.

Good luck fellas!

 

* I excluded all data from this post that can be attributed to any companies I worked or I'm working for at the moment. Names, characters, numbers, charts, findings, insights and incidents either are products of my imagination or are used fictitiously. Any resemblance to actual data or persons, living or dead, is entirely coincidental. Findmyho.me does not exist, I am not an expert of the housing market and I don't recommend you to start a business based on the data or insights presented in this post.

(not so) evil retention drivers

User retention is important. It is essentially the number that shows you how many users come back to your product in a given timeframe. If it's high, you're happy, if it's low you're sad. You would like to increase it over time. Just like this graph shows it.

 
 

It is so obvious that everyone in the business would like to increase this, that most of the people don't really ask why. So what does this number really encapsulate?

User retention is one number out of many that tells you how valuable you're product is. And by value I mean user value and not business value. Ideally anytime a person interacts with your product s/he does it because s/he would like to get value from the product.

 
 

If they come back more often this means that they find your product valuable. The more often they interact with it the more user value they get. Clear. This means that if your product becomes more valuable the higher your retention number will be.

Ok. So let me tell you what's happening nowadays in the product/growth world. As I wrote about it in this post, many people and businesses confuse user value with business value and thinking about creating business value first.

This means that they look at retention purely as a business value indicator. If more people come back the more business value is generated on our side. Don't get me wrong, this is 100% true. And this is very important too. The problem is the mindset. If you look at it from this perspective then your natural question is how to bring back more people to capture more business value. Instead of this if you embrace the "User Value First" mindset you will ask how to create more user value in the product, so people will eventually come back more often. (Btw, this will obviously generate business value.)

To illustrate the difference here's a graph:

 
 

As you can see the two different mindsets will result in two different retention drivers.

  1. (user) Value As Retention Driver (VARD)
  2. (retargeting) Notification As Retention Driver (NARD)

VARD

VARD is powered by user needs, the "jobs-to-be-done", the triggers those are bringing people back. If these triggers are happening and people think that they can get the job done using your product they will come back. So the more user value you create by solving real user's needs the stronger your VARD will be. And a powerful VARD moves retention rate higher.

NARD

NARD is powered by the notifications you send to people. If you send more notifications the stronger your NARD will be. The problem is that NARDs can easily piss off people. They get too much useless notifications, they are distracted with the latest promotions, ads, etc. So basically they are not getting value from the product.

It is easier to set up a quick NARD, than to improve the VARD. And as I said above it is a mindset question too.

The good news is that there are good NARDs too. The trick is that you need to find what's in the intersection of a NARD and VARD:

 
 

You need to make a shift in your mindset and ask how can you create value for the user with notifications?

I always say a good notification is the one that arrives exactly at the moment when the user is secretly dreaming about it. Such notifications are:

  • a friend sends you a message
  • a friend starts following you
  • you have set a price alert to notify you when prices go down
  • traffic jam ahead of you alert showing an alternative route
  • etc.

These are the good NARDs. You will create user value if you create these. And if you do this, trust me, your retention number will be higher and your users will love and use your product much more. :)

Good luck fellas!

 

PS. There are tons of awesome posts out there talking about the various retention types in detail. I didn't list them here, because it would have distracted us from the main topic. Probably I'll do it in another post. I especially like working with return retention, that is basically the proportion of users coming back to the product at least in N days.

 

the corporate board game and the gold rush

Ok. So you're a Product Manager or Product Owner.

I bet 99.9% of the answers are B. So why is it like that?

In every organisation all employees are part of a huge corporate board game. It's a board game about occupying valuable territories. Imagine a huge board with different territories on it:

 
 

Certain territories in this board game have more valuable raw materials. Players would like to occupy these territories because they're worth more. In case of a product company the most valuable territory is the Product. It is the core of the business. It is the engine. It drives the business. It generates the user value and the business value too. If it wasn't there then you're business wouldn't exist at all.

All ambitious players in the game would like to occupy that area. This is totally normal. Everyone is a player, a gold miner. And you know it's a constant gold rush happening. Each player has it's own bet where the gold is exactly. They just need to go there and tell where to dig. Then they'll have it and they're going to be rich and famous. They would like to have the whole area digged, because they're so sure that there is gold. Well, this is how a feverish gold rush works. No offence. There is tremendously huge energy in a gold rush. And there is the chance that there is gold actually. But it's just hard to manage this energy.

 
 Source: [1]

Source: [1]

 

And you know there are these guys on that territory. They own and manage the territory. Hence they are called Product Managers or Product Owners. Their single most important job is to discover gold. Discover. Gold. They can even do it in a well-orchestrated way. They have these awesome tools they can apply. Like a metal detector (researchers) for example. And a full crew of people who can dig large holes in the ground (engineers). They've already mined several times and also discovered gold many times (experience). They have tools to dig very fast if they want. They can do exploratory drilling (A/B testing). They can ask people bring up gold. And they even work with a lot of goldsmithes to create beautiful jewels (designers). They are just, you know... experts.

So what happens is that all the other people in the business start playing this board game. The biggest problem is that usually they don't pick up a shovel and a spade. They rather point to various spots on the field and tell the product manager that they should really dig there. And you know if it's a gold rush, it's a feverish belief. And it's really hard to influence or convince them that there is another way to do this. They will keep saying that there is the spot they think you need to dig until you give up.

So what can you do as a Product Manager? You have multiple options:

  1. Close the border
  2. @#&! Quit the board game
  3. Create a proper administration system
  4. Teach people how to discover gold and even offer them tools to do this

I have to tell you, I've tried all of these. Here are the pros and cons of each:

Close the border

 
 Source: [2]

Source: [2]

 

Pros: Easy. Doesn't cost too much.

Cons: It creates a lot of frustration. Riots will happen. Some people with a panzer (business executives) will enter anyhow.

@#&! Quit the board game

 
 

Pros: Easiest thing. Doesn't cost anything.

Cons: You don't really face the problem. You will feel it's a failure, you will be frustrated for the rest of your life because you know that this is frikin' Alaska and there is gold. But you couldn't manage to discover it all.

Create a proper administration system

 
 

Pros: It can work. You just need to create a very smooth system and educate people how to use it and what to expect from it.

Cons: It's easy to make this a very bureaucratic system. You can easily die in doing the paper work. Even some assumptions that could have been good could be overlooked. Some people simply don't like systems. They will complain and try to break it anyhow.

Teach people how to discover gold and even offer them tools to do this

 
 

Pros: You gain a lot of new educated workers. They understand the methods and techniques. More gold will be mined eventually. They can even teach others how to do this. The territory becomes a boosting economy.

Cons: It takes hell a lot of time and effort. If you do this then you won't have enough time to focus on the discovery. Some of your students will eventually say:  "Those who can, do; those who can't, teach."

I'm sorry, but there is no clear answer. You need to assess your organisation first. Based on this I recommend you to find the balance between #3 and #4.

Unfortunately you also need to do some politics too. Sorry. This means that you need to practice #4 and educate people first who own a panzer. If you do this then they will be on your side and support you. If you don't do this, you will eventually fallback to #2, which is no good.

In terms of the administration system I strongly recommend to set up a proper Opportunity Backlog (I will blog about this later) and teach people how to add new opportunities to it properly. It can work very well, but at the point you feel that there is more and more bureaucracy then do more from #4.

And well... this was just people from other fields. Riots can happen on the Product field too. But let's talk about that later :)

Good luck fellas!

 

Clip art sources:

[1] Clip art image by ClipArtHut: http://www.cliparthut.com/gold-nugget-clip-art-clipart-cxvgg2.html

[2] Clip art image by Clipart.co: http://cliparts.co/clipart/2623811

create value - but what #HowToManageAProduct

I have worked on products with quite a few teams and people. Engineers, designers, sales people, business executives, CEOs, other product managers, just to mention some of them. While working with these people I learned a lot. I learned a lot about their aspirations, views, visions, goals and execution methods too. I engaged in a lot of thoughtful discussions, plenty of debates and decision-making processes too.

After many debates I noticed something. It may look like a tiny thing. But it is huge. In the world of products there is this very important concept that is almost always misunderstood. It is not even just misunderstood but also biased by personal preferences.

This concept is the value you create.

Why is this concept so important?

Some phrases that I often hear when people are arguing about products:

  • "we need to create value"
  • "our goal is to create value"
  • "is this really valuable?"
  • "let's create value for the user"
  • "there is no value in this feature"

Don't get me wrong. I'm also using these in many cases. The problem is that we should only use "value" as an argument if every one of us who's in the discussion interprets it the same way. Otherwise we're just trying to use a word that sounds great and hides our intentions.

According to Marty Cagan we should create products those are valuable to the customers and to the company, usable to the people who want use it to get value and feasible to build given the time and tools that you have.

Unfortunately most of the people translate it this way: "Let's create a feature in 1 month that is nicely designed and people would pay for it."

Well in this case things are lost in translation.

What does "value" mean then?

I won't give you here an economics-focused definition. Neither I won't go into philosophic details. I will leave it to you to do that. Value is a very hard concept in itself and it would be hard to say that this is the absolutely correct definition.

One thing is sure: many people confuse user value with business value. And this can be deadly.

They are connected but they are not one and the same things. Of course you need to look after both if your job is connected to a product. But you always need to be able to distinguish the two if you're a product manager.

User value: The value that a user gets from the product. (Tweet this)

Business value: The value the business gets if the user is using the product. (Tweet this)

This shows the difference. You need to create value for the user first and then you can capture that for your business. If you do it the other way then you're going to die as a business. The reason is simple. Your existence depends on the people who use your product. (Tweet this) Therefore if you don't create value for them they will leave you. So when there is a debate about if a feature creates value or not, then please clarify that you're talking about the user value first.

 
 

User value

A practical question can be: how can I identify user value? There are multiple techniques to do this and I will write about in another post, but let me help you a little here too. I will talk about three basic models.

  1. Find User Problems
  2. Find Jobs To Be Done
  3. Find Magic Moments

Find User Problems

The most commonly known theory/model is that if you solve a user problem then you create value. Hence the basic technique is that if you know that X is a user problem and you create Y as a solution (e.g. a cool feature) then you create value. Easy. The hard thing in this is that user problems are not crystal clear in many cases. There is an interesting thought that there are the Painkillers and the Vitamins. Obviously painkillers are solutions for real problems, while vitamins are nice to have. Although this may sound appealing if I would ask you to define the user problem that Facebook or Twitter solves, it would be hard to come up with a "painkilling" phrase that explains why over 1 billion people use them. Anyway using this technique can work. And I suggest not to ban it entirely in discussions because it's a common language many people speak.

Find Jobs To Be Done

Another theory/model that is getting more popular and I like it very much is the Jobs-To-Be-Done model. It was first introduced by Clayton Christensen. Here you can find a great collection of articles about the framework and a lot of connected techniques. The main concept is that people "hire" products to do a "job" they want to be done. As legendary Harvard Business School marketing professor Theodore Levitt put it, "People don't want to buy a quarter-inch drill. They want a quarter-inch hole!" Based on the model the only thing we need to do is to discover these jobs. This essentially means we need to look for the needs.

One thing is common in these two models. And that is you need to discover the needs/problems. (Tweet this) You need to know that the solution you are building is a valuable product. This leads us to Get Out Of the Building.

Find Magic Moments

Let me give you here a third theory/model. I really like it too and it is very much related to the discovery aspect of product management. It is coming from the good Tom Chi, who's the Experience Lead at Google X. He essentially says that you need to find "magic moments" with your user. (Tweet this) These beautiful moments are actually what your product is all about. "Your product is not the search box, the ad algorithm, your user records – it is this magic moment, and your task is to get out of the way of the magic moment or to amplify the magic moment." (Martin Eriksson's transcript) The only way to identify these magic moments is to talk with people and show your product/feature/prototype to them. If you find a magic moment, that is the user value.

This basically equals with the 80/20 rule. The real user value in your product is that 20%. So anytime you're in a debate with someone if this feature is valuable or not, please ask if that feature would create a "magic moment" or not. Remember though that anything what you say is just a conjecture. It's not reality unless you have built at least a prototype and have shown it to a user and have some data from that test. Facts are the only reality. The same goes for magic moments.

Business value

Business value is not always revenue. (Tweet this) This is important. Let me illustrate this and also the controversy between user value and business value with an example.

Let's say you have an e-commerce product. People can use this product without signing up with their email addresses or Facebook accounts. But if they sign up then you can have their email addresses, you can retarget them, bring them back to the site so they can use it more and eventually they will probably buy more. Obviously it means business value for you.

There are two mindsets how you can approach the situation. These mindsets are significantly different. Here are two quotes those illustrate them.

  1. "Business Value First" Mindset: "How to make people signing up more? Should we make sign-up mandatory to have more revenue eventually?"
  2. "User Value First" Mindset: "How can I create value for the user by solving a user need? What problem can I solve where I need the user's identity?"

"Business Value First" Mindset

"How to make people signing up more? Should we make sign up mandatory to have more revenue eventually?"

It's very likely that you will come up with a solution what I call artificial user value generation. That is, you start limiting your product, by limiting one of your features. You create necessity. A typical action created from this is: change the product so that a user can only use the advanced search filter if s/he signs up. If you do this you don't actually create user value, you just create necessity.

"User Value First" Mindset

"How can I create value for the user by solving a user need? What problem can I solve where I need the user's sign-up?"

Apparently this is a tougher question to answer. But in the answer there is the real user value. It is very likely that you will come up with a feature that is something like personalised recommendations. You aren't able to create this feature without getting the identity of the user. If you do this on your e-commerce site you're likely create user value. (This feature is just an idea. I'm not 100% sure about the solution because I haven't tested it. Based on what Amazon does this seems like an obvious thing though.)

Don't get me wrong. Business value is not from evil. It is necessary. It is the gasoline you need to have to get from A to B with your car. But your real goal is to get to B and not to pile up gasoline.

Conclusion

In this post I wanted to clarify the main difference between user value and business value. User value is the value that the user gets from the product. Business value is what the business gets when the user is using the product. Many people confuse this and because of different personal goals in a business it often leads to biased decisions. Please try to clarify this anytime you feel that you're not having the same goals when debating about a new feature. It is crucial.

There are two things I would like to leave you with:

  1. Look for the user value first. After that try to capture the business value. Business value is the gasoline in your car but the real goal is to get to B and not to pile up gasoline. (Tweet this)
  2. If you're working on a product, please ask: "What is the magic moment in my product?" (Tweet this)


product management - the definition-paralysis

When I started my first job as a product manager I only had a vague idea what product management means. My first thoughts were that basically the product manager is a project manager who is responsible to deliver a product in the end. I also read some thoughts that a product manager is a mini-CEO of a team, and the team is like a startup in a company. Then some people said that product management is about business and marketing. It was really confusing. Anyway since I love building products and had some management experience it all sounded awesome.

It is still a widely debated thing what exactly product management is. Multitudes of definitions, many writings are out there. I especially like the description written by Ben Horowitz. It gives you a sense about product management. I'm also the product manager who leverages the thinking of Marty Cagan. He considers the responsibility of a product manager to create a product that's valuable to the customers and the company, usable to the people who want use it to get value and feasible to build given the time and tools that you have.

  I usually use this basic visualisation

I usually use this basic visualisation

There are some questions though that this definition raises. Like what does it mean to create a product, is there a team in the process, is there anything that's valuable but not usable, etc.

I attended a conference once where people spent hours with discussing what product management is. And though it's an interesting philosophical question I consider product management a continuously evolving, constantly changing role which is highly dependent on the organisation. I got angry after a while during the conference, because we fell into some kind of definition-paralysis. Totally unproductive.

So I consciously don't define here what product management is. I love the description of Marty Cagan, Ben Horowitz and other gurus like Steven Haines. I strongly argue that product management is about getting things done. So please don't spend too much time trying to define it.

Why did I put this here?

I put it here because I want to deal with real issues in my blog. Issues that product managers have. I would like to explain how I handle them and what my learnings are from my experiences. If I might fall into the trap of being to philosophical, please feel free to kick my butt.

With these closing thoughts I start a series (How to manage a product?). I will write about the things those are necessary to be done by product managers. I will give recommendations how to handle certain situations. One thing is sure: The goal is to help you build products that people will love and use.