My summary of “Winning with Data”
Tom Tunguz, a venture capitalist at Redpoint Ventures, and Frank Bien, CEO of Looker, got together to make an incredible book about how data is transforming business organizations.
I think we all understand anecdotally the amount of data the streams off of digital products and services. Every action on a website. Every button pressed. Every preference set. This should all be saved by companies.
The most famous examples — Facebook and Google — are relentless in testing new products and features, and following the data.
It’s reasonable to assume an e-commerce company records each user’s actions. The team should know that User A signed up and purchased an item while User B signed up, browsed for 15 minutes, and put an item in his/her cart but never purchased.
These two simplified examples show what is possible when a company builds a product and infrastructure around tracking events and storing events. In the first case, perhaps we should send a Thank You Email and update the user when we have an estimated arrival date.
For User B, perhaps we should send an email (collected on sign up) to remind them to purchase the item. Or maybe we can provide a 10% off coupon next time they visit the site. These are the rather obvious consequences of data collection.
But Tunguz and Bien go much, much further. The most important of their book in my opinion are the steps needed to take full advantage of data. Most changes reside at the organizational level, not the product level.
I highly recommend this book, particularly to those with access to large data sets or those in unproductive organizations. Data is great for moving things forward because it doesn’t care about your opinion.
Have someone dedicate some of their time to data
There needs to be at least one person who has the responsibility of looking through and understanding data.
If you have different people jump in and out, they will largely uncover the same information from your data. But if the same person spends his/her time looking at data for long enough, insights will come.
And you will begin to see the holes. Someone who digs one level deeper than surface level will uncover flaws in how you are storing, accessing, and analyzing data. This is crucial.
Create and circulate a data dictionary
You need a “source of truth” for definitions. Even something as simple as “What defines a user?” is not entirely agreed upon as teams grow. Talk to a company that has 20 or 30 or 40 employees, and you’ll see what I mean.
Going to back to the e-commerce example, does someone become a user when they enter their email and password? When they confirm their email? Or is it when they add a credit card and mailing address?
Similarly, is a monthly active user someone who logs in and browses on your website within the last 30 days — or do they need to buy something to be counted in the monthly active bucket?
In other words, you need to get everyone on the same page in your team, because when you want to improve “onboarding” or “user retention” your different teams need to understand explicitly what that means.
Write everything down in a public place and make sure everyone has access to it. Part of a new employee’s training should be to review this document and ask questions.
Turn data into action items
Most organizations spend their resources on data collection — and this is problematic because the collection of data is a means to an end. If data collection is accounting, then data analysis is finance.
You need to build a culture where people have the ability, and tools, to turn that data into actionable items. Let’s say you want to understand the funnel from account creation to purchase, then improve it.
In your app, the flow is: Download from App Store, create an account, and purchase an item. Right now, 70% of people who download the app create an account. And 30% of people who create an account successfully purchase something.
That means we have a 21% success rate (70% * 30%). Our goal is now to improve this rate by effecting either account creation or check outs. Given our resources, we can only improve one. Which should we focus on?
Well, let’s improve both by 5% and see what happens.
Baseline: 70% * 30% = 21% success
Option A: 75% * 30% = 22.5% success (7.1% improvement over baseline)
Option B: 70% * 35% = 24.5% success (16.6% improvement over baseline)
As a rule, teams should spend their time improving the worst converting part of their funnel because that is where the largest gains are found.
If you’re still having trouble with this, it may be helpful to think of the extremes: The change from 1% to 2% doubles your outcome, while going from 98% to 99% will not even be noticed.
Dedicate time for office hours
People in organizations always have questions. In fact, answering one question may lead to two or more new questions. And given the complexity of modern organizations, people can either not have the time to surface these questions or they may not know where to start when asking.
Office hours help solve these problems by creating some predictability and structure. People can bring any question — even one of that is outside of an employee’s day-to-day job — and ask the experts.
In small organizations, there may be one expert. In a publicly traded company, there could be dozens of people on a data team. Regardless, this exercise helps people become more familiar with company data.
After office hours, the questions and answers should be posted in a public, easily accessible place. This way a repository is built of what was asked, how it was stepped through, and the results that were found.
Office hours provide a time and place where everyone is working towards solving the problems presented. Again, it is important to focus office hour sessions around an action or solving a company problem.
